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Researching education, improving learning 


Aligning official curricular intentions with teacher instructional 
decisions in the classroom is widely believed to lead to 
improved student performance. 

As active agents and decision makers in this process, rather 
than as passive deliverers of prescribed curricular content, 
teachers make significantly different instructional decisions in 
enacting the curriculum. 

In TIMSS, teachers in high-achieving education systems 
reported teaching more challenging mathematics content; 
teachers in low-achieving education systems reported 
spending more time teaching mathematics, and had high 
performance expectations. 

In all education systems participating in TIMSS, teachers’ 
enactment of a mathematics curriculum diverged considerably 
from official intentions. 

Curriculum- and teacher-related policies should focus on ways 
to enable teachers to better adapt the intended curriculum to 
specific contexts and student populations. 


IEA COMPASS 


BRIEFS INEDUCATION 


NUMBER 6 JUNE 2019 


Setting high standards for the overall 
mathematics curriculum and ensuring teachers 
adhere to these standards may not necessarily 
lead to better learning outcomes. 

Mathematics performance in TIMSS appears 
to be independent of teachers’ instructional 
alignment with the official curriculum. 

The analyses show significant gaps between the 
intended and implemented grade 8 mathematics 
curricula in many TIMSS education systems, 
regardless of performance. 

Education systems should foster conditions that 
enable teachers to better enact the intended 
mathematics curriculum, taking into account 
students’ previous knowledge and current 
needs. 

teacher 


Policies regarding 


mentoring, accountability, and evaluation, 


supervision, 


should support teacher enactment of the 
curriculum. 


@ IEA COMPASS 


INTRODUCTION 


As curriculum enactors, teachers actively engage in making 
decisions about the prescribed curriculum. In doing so, they 
create opportunities to learn that vary by classroom, school, and 
system. Understanding teacher decisions is key to improving 
student performance. While many policies are designed to 
influence teacher pedagogy, particularly their alignment with 
curricular intentions, research continues to focus on how and 
why teachers implement the curriculum in particular ways. 
Especially important are studies that examine how instructional 
practices differ within and between education systems, and 
with what impact on learning. 


The IEA’s Trends in International Mathematics and Science 
Study (TIMSS; see https://www.iea.nl) provides an excellent 
comparative platform to explore variation in teacher 
instructional practices (Mullis & Martin, 2013). This Brief 
examines the mathematics content teachers actually teach 
during the grade 8 in low-performing and high-performing 
education systems and focuses on two critical questions: To 
what extent do teachers provide significantly different learning 
opportunities to students in low and high performing education 
systems? To what extent are these differences related to the 
degree to which teachers align their instructional practices with 
the prescribed curriculum? 


DATA AND KEY MEASURES 


After assessing their average student results for grade 8 
mathematics in TIMSS 2007, 2011, and 2015 (Mullis et al., 
2008, 2012, 2016), we selected and categorized 15 education 
systems as either low or high performers in mathematics (see 
Appendix for details). An education system was categorized as 
high performing if their average grade 8 mathematics score in 
the three TIMSS cycles was above the TIMSS scale center point 
(500 points), and low performing if it was below the TIMSS scale 
center point. 


We examined three measures of the curriculum: content topics, 
instructional strategies, and instructional time (see Appendix 
for further details). For each measure we calculated the 
percentage of teachers that covered the mathematics content 
topics, or employed the instructional strategies, and the average 
percentage of instructional time devoted to the teaching of 
mathematics by education system. Measures for the enacted 
curriculum were taken from teacher reported data. While this 
is the best data source available, the surveyed teachers do not 
constitute a representative sample of all teachers. TIMSS 2015 
sampled and tested grade 8 students and the teachers surveyed 
were their mathematics teachers and not a sample of all grade 8 
teachers (Mullis & Martin, 2013). 


All three measures of the enacted curriculum varied across 
and within education systems. In other words, the analyses 
did not identify distinctively different patterns of learning 
opportunities among either the low- or high-performing 
education systems. 


CONTENT TOPICS 

Content topics taught either prior to or during grade 8 were 
reported by teachers, and were based on the 18 content topics 
across four domains of mathematics included inthe TIMSS 2015 
framework (Mullis & Martin, 2013). We found that, in general, 
teachers in both low- and high-performing education systems 
enact quite similar mathematics curricula, covering topics from 
all four content domains: number, algebra, geometry and data 
(see Figure 1). In all education systems, teachers reported high 
coverage of number topics, which are usually taught from the 
early grades of primary education. By contrast, there were 
important differences in the coverage of algebra and geometry 
topics, especially the more advanced topics that are often 
taught in higher grades. For each content domain, a lower 
percentage of teachers in low-performing education systems 
reported covering more complex content topics, with two 
exceptions: concepts of irrational numbers, and translation, 
reflection and rotation. It should be noted, however, that the 
latter content involves three subtopics that may be covered in 
different grades. Reflection, for example, is often taught earlier 
in the schooling progression. Thus, for this particular grouping 
in the questionnaire, teacher reports may be less accurate. In 
general, teachers in both low- and high-performing education 
systems reported covering algebra, geometry and data topics 
less extensively than number topics. 


At the system level, the grade at which topics are introduced, 
the number of topics covered per grade, and the consistency 
with which topics are covered from grades 1 to 8 varied across 
education systems. 
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Figure 1: Percentage of teachers who reported teaching mathematics topics during or before grade 8 in both low- and high-performing 
education systems 
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INSTRUCTIONAL STRATEGIES 

As in previous cycles (2007 and 2011), teachers in the 2015 
cycle of TIMSS reported how often they employed specific 
instructional activities in their classroom teaching. Questions 
relate to how teachers explain new mathematics content, 
whether students are asked to memorize rules, procedures, 
and facts, and whether students decide on their own 
problem-solving procedures. From these reports we created 
two variables by categorizing activities by the kinds of the 
cognitive processes they entail (see Appendix): we term these 
as (1) less challenging instructional strategies (LCIS) and (2) 
more challenging instructional strategies (MCIS). Activities 
associated with remembering, which is considered to be one 
of the more basic processes according to Bloom’s revised 
taxonomy (Krathwohl, 2002), were classified as LCIS, whereas 
more analytical, interpretative or communicative processes 
were categorized as MCIS. We then calculated the percentage 
of teachers who emphasized to a lesser or greater extent each 
type of instructional strategy.* 


In all education systems, almost all teachers reported exposing 
students (with moderate or high frequency) to more challenging 
instructional strategies (Figure 2). The use of less challenging 
instructional strategies was less prevalent and varied by 
system. Perhaps surprisingly, 57.9 percent of teachers in low- 
performing education systems reported frequently covering 
MCIS, against only 44.4 percent of teachers in high-performing 
education systems. Conversely, 42.9 percent of teachers 
in high-performing education systems and 29.2 percent of 
teachers in low-performing systems reported moderate to high 
coverage of LCIS. 


1 The response categories for each item were: (a) every or almost 


every lesson; (b) about half of the lessons; (c) some lessons; and 
(d) never. We then averaged the responses for each type of 
instructional strategy to determine whether the emphasis on that 


strategy was weak, medium or strong. 


Figure 2 : Percentage of teachers that exposed students with moderate 
or high frequency to more challenging (MCIS) and less challenging 
(LCIS) instructional strategies 
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ENACTED INSTRUCTIONAL TIME 

Actual instructional time was measured by the number of 
hours of mathematics instruction during the year, according to 
teacher and school questionnaire data. The analysis shows that 
annual instructional time spent on mathematics in the low- 
performing group of education systems varied considerably 
more between the 25th and 75th percentiles than that in the 
high-performing education systems? (Figure 3). Within each 
group of education systems, instructional time also differed 
significantly. Overall, teachers reported that the opportunity 
to learn mathematics in relation to instructional time varied 
significantly across education systems, from between 40 or 50 
hours per year, to 220 hours per year. 


ries 
2 Outliers (< 5 percentile points) in both groups were eliminated. 
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Figure 3: Distribution of annual mathematics instructional time in low- and high-performing education systems 
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INSTRUCTIONAL ALIGNMENT 
To compare the intended curricula with the enacted curricula, 


we analyzed content and instructional time data reported in 
the TIMSS 2015 curriculum questionnaire. For each education 
system and mathematics topic, we calculated the percentage 
of teachers who reported covering the topic, and related 
such percentages to the inclusion or non-inclusion of these 
topics in the intended grade 8 curricula. The intended and 
enacted instructional times were compared using measures 
of the percentage of time allocated and devoted to teaching 
mathematics out of the total available annual instructional time. 
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In terms of content, both low- and high-performing education 
systems showed important gaps between the intended and 
enacted curricula. Although the average percentage difference 
between these was larger for the low-performing group, 250 
percent of the teachers surveyed in both groups diverged 
from the intended curriculum for some of the topics, and this 
was true for all content domains. High-performing education 
systems, such as England, Hong Kong, the Republic of Korea, 
Singapore, and the United States displayed differences in three 
or more content domains (Table 1). 


In both groups, there were some education systems where 
<50 percent of teachers covered the intended number topics. 
However, topics that were not included in the official curricula 
were taught by a considerable percentage of teachers in both 
groups of education systems. For example, in the case of 
algebra topics, the average percentage of teachers in the low- 
performing group who reported teaching topics that were not 
included in the official curricula was similar to that in the high- 
performing group (29.7% and 24.7%, respectively). 


IEA C@MPASS 


Meanwhile, 60.3 percent of teachers in high-performing 
education systems but only 49.6 percent of teachers in low- 
performing countries reported covering intended algebra 
topics. 


In the content domain of geometry, although the average 
coverage of non-prescribed topics was higher in low-performing 
(36.1%) than in high-performing education systems (22.9%), 
across both groups around 50 percent of teachers reported 
covering intended topics. 


Table 1: Education systems with at least two content topics in each content domain, reporting = 30 percentage point differences between the 


grade 8 intended and the enacted curricula 


Number Chile, Iran, Jordan, Lebanon, Malaysia, Thailand 
Chile, Iran, Jordan, Lebanon, Malaysia, Thailand, 
Algebra 
Turkey 
Chile, Iran, Jordan, Lebanon, Malaysia, Thailand, 
Geometry 


Turkey 


Data and chance 


We compared intended and enacted instructional time by 
plotting the percentage of annual instructional time actually 
allocated to teaching mathematics against the corresponding 
curricular intentions for each education system (Figures 4 and 
5), as well as determining national averages (Table 2). Although 
we observed considerable variation within groups, low- 


Iran, Jordan, Malaysia, Turkey 


Chinese Taipei, England, Hong Kong, Hungary, 
Republic of Korea, United States 


England, Hong Kong, Hungary, Japan, Republic of 
Korea, Singapore, United States 


England, Hong Kong, Hungary, Republic of Korea, 
Singapore, United States 


England, Hungary, Singapore 


performing education systems, such as Chile, Lebanon and Iran, 
showed more variation in terms of the enacted instructional 
time. On average, the variation in the low-performing group 
was considerably greater than in high-performing education 
systems, indicating greater divergence from the intended 
curricula. 
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Figure 4: Time dedicated to mathematics instruction as a percentage of total annual instructional time in high-performing education systems 
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Figure 5: Time dedicated to mathematics instruction as a percentage of total annual instructional time in low-performing education systems 
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Table 2: Average intended versus enacted instructional time in low- and high-performing education systems 


EDUCATION SYSTEM 


ONAN) = WAY NC] 1 AVERAGE VARIANCE 


ON MATHEMATICS ALLOCATED TO WITHIN THE 
INSTRUCTION AS MATHEMATICS EDUCATION SYSTEM 
A PERCENTAGE INSTRUCTION AS 6/4) 
OF TOTAL A PERCENTAGE 
INSTRUCTIONAL TIME OF TOTAL 
6/4) INSTRUCTIONAL TIME 
ie) 
Chile 16.6 12.0 16.5 
Iran 12.5 13.0 14.4 
Jordan 13.7 15.0 8.7 
Lebanon 16.7 16.0 17.3 
Low-performing 
education systems 
Malaysia 10.9 12.0 9.8 
Thailand 7) 10.0 5.0 
Turkey 12.1 13.0 15.8 
Average 13.1 13.0 12.5 
Chinese Taipei 14.2 12.5 6.2 
Hong Kong 13.8 13.5 7.6 
Hungary 13.3 12.5 8.4 
lle estntel ins Japan 10.1 10.0 oa 
education systems 
Republic of Korea 12.1 11.0 3.0 
Singapore 12.3 15.5 6.5 
Average 12.6 12.5 5.8 


We found that in both low- and high-performing education 
systems, teachers diverged considerably from the intended 
curriculum. However, some high-performing education 
systems, particularly Hong Kong, Japan, and the Republic of 
Korea, followed instructional time expectations more closely. 


It is difficult to explain the low correspondence between 
intended and actual instructional time. It may be that teachers 


are exposed to more diverse student populations, and that 
teachers, as decision makers, thus adjust their instructional 
strategies to cover particular elements of the mathematics 
curriculum. It may be that schools reflect different local needs 
(for example, automatic progression, or low retention levels) 
and teachers are expected to provide the response. These, and 
other explanations, while plausible, merit further scrutiny. 
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CONCLUSIONS AND POLICY IMPLICATIONS 


Our analyses have demonstrated that significant gaps exist between the intended and implemented grade 8 
mathematics curriculum in many education systems, demonstrating considerable variation between official curricula 
and what teachers actually teach. These is true of both low- and high-performing education systems, and suggests 
that high-performance in mathematics is not necessarily dependent on teachers fully enacting or aligning their 
instruction according to the prescribed contents of the official curriculum, as many researchers and policymakers 
have assumed. Setting high standards for the overall national mathematics curriculum and ensuring that teacher 
adhere to these standards does not necessarily lead to higher learning. 


The key challenge for education systems is to identify and foster conditions that enable teachers to better implement 
the intended mathematics curriculum. For example, this might encompass providing professional development 
opportunities so teachers can better assess and respond to student learning needs, or helping teachers to adapt 
their teaching strategies and available resources to achieve more meaningful instruction. Teacher policies regarding 
teacher supervision, mentoring, accountability, and evaluation, should also be aligned with this challenge in mind. 
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DATA AND KEY MEASURES 


We selected education systems that participated in three cycles of TIMSS (2007, 2011, and 2015) and that for two of three 
cycles had a grade 8 average mathematics score either significantly above or below the TIMSS scale center point. We also 
considered including education systems from different continents and selected those for which there was data for the variables 
used in the study. Fifteen countries met the criteria. These were categorized as low- and high-performing education systems. 
Education systems with average results significantly above the TIMSS scale center point (500 points) were categorized as high 
performing, while education systems with mean scores significantly below the international scale center point were categorized 
as low performing systems. The education systems that we included in our study were: Chinese Taipei, England, Hong Kong SAR, 
Hungary, Japan, thr Republic of Korea, Singapore, and the United States (the high-performing group), and Chile, Iran, Jordan, 
Lebanon, Malaysia, Thailand, and Turkey (the low-performing group). 


National experts reported data for curricular intentions, while teachers reported information on the implemented curriculum (for 
further information on the TIMSS 2015 data please consult the IEA data repository, available open-access at www.iea.nl/data). 


Content topics covered four domains: numbers, algebra, geometry, and data and chance. We analyzed a total of 20 subtopics 
within each of these domains. 


Differences in the mean coverage of content topics and instructional strategies were tested using a Pearson chi-square test. 
We used the following items to construct the instructional strategies variables: 


e LESS CHALLENGING INSTRUCTIONAL STRATEGIES (LCIS): listening to the teacher explain new mathematics 
content; listening to the teacher explain how to solve problems; asking students to memorize rules, procedures and facts; and 
working on problems with the teacher’s guidance. 


e MORE CHALLENGING INSTRUCTIONAL STRATEGIES (MCIS): working on problems with the whole class and 
guidance from the teacher; relating the lesson to students’ daily lives; asking the students to explain their answers; asking the 
students to complete challenging exercises that require them to go beyond the instruction; asking the students to decide their 
own problem solving procedures; and working on problems for which there is no immediately obvious method of solution. 
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